Systems and methods for determining projected target location of a handheld object

ABSTRACT

A projected target location of a handheld object is determined based on applying translation factors, scaling factors, and offsets to a location of a reference element of the handheld object detected by a camera on a two-dimensional plane. The translation factors are determined based on a difference between a calibration location on the plane and an initial location of the reference element corresponding to the calibration location, and serve to shift the location of the reference element to generate the projected target location. The scaling factors are determined based on an estimated length of a user&#39;s arm holding the handheld object, and serve to scale the location of the reference element to generate the projected target location. The offsets are determined based on polynomial equations, and serve to extend the distance between the projected target location and the calibration location.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 17/364,241, entitled “Systems and Methods for DeterminingProjected Target Location of a Handheld Object,” filed on Jun. 30, 2021,which is a continuation of U.S. Non-Provisional application Ser. No.16/749,865, now U.S. Pat. No. 11,100,790, entitled “Systems and Methodsfor Determining Projected Target Location of a Handheld Object,” filedon Jan. 22, 2020, which claims priority from and the benefit of U.S.Provisional Application No. 62/905,901, entitled “Systems and Methodsfor Determining Projected Target Location of a Handheld Object,” filedon Sep. 25, 2019, each of which is hereby incorporated by reference inits entirety for all purposes.

BACKGROUND

The present disclosure relates generally to a handheld object used forpointing and, more particularly, to determining a projected targetlocation of the handheld object.

This section is intended to introduce the reader to various aspects ofart that may be related to various aspects of the present disclosure,which are described and/or claimed below. This discussion is believed tohelp provide the reader with background information to facilitate abetter understanding of the various aspects of the present disclosure.Accordingly, it is understood that these statements are to be read inthis light, and not as admissions of prior art.

A handheld object may be used to point at or select a target. Forexample, in the setting of a theme park, a patron may point at ananimated figure of an attraction using the handheld object, and, inresponse to detecting this, a system may cause the animated figure tooutput a user interaction experience (e.g., wagging a tail). However, itis now recognized that certain physical characteristics relating to theuser's body may present difficulties in accurately determining when theuser is pointing at the target.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a diagram of a user pointing a handheld object at a target,according to an embodiment of the present disclosure;

FIG. 2 is a block diagram of a theme park attraction system, accordingto embodiments of the present disclosure;

FIG. 3 is a diagram of a user pointing a handheld object at acalibration location, according to an embodiment of the presentdisclosure;

FIG. 4 is a diagram of an example of applying one or more translationfactors to a subsequent detected location of a reference element of thehandheld object of FIG. 3, according to embodiments of the presentdisclosure;

FIG. 5 is a diagram of an example of applying a scaling factors to asubsequent detected location of a reference element of the handheldobject of FIG. 3, according to embodiments of the present disclosure;

FIG. 6 is a diagram of a user pointing the handheld object at differenttargets of a system according to embodiments of the present disclosure;

FIG. 7 is a diagram of differently-sized multiple reference elementzones and uniformly-sized multiple projected target zones, according toembodiments of the present disclosure;

FIG. 8 is a diagram of uniformly-sized multiple reference element zonesand differently-sized multiple projected target zones, according toembodiments of the present disclosure;

FIG. 9 is a flow diagram of a process for determining a projected targetlocation of the handheld object of FIG. 3, according to embodiments ofthe present disclosure; and

FIG. 10 is a flow diagram of a process for compensating for thedistortion caused by a difference in shape between an arcuate nature ofa user's arm movement and a two-dimensional plane, according toembodiments of the present disclosure.

DETAILED DESCRIPTION

One or more specific embodiments will be described below. In an effortto provide a concise description of these embodiments, not all featuresof an actual implementation are described in the specification. Itshould be appreciated that in the development of any such actualimplementation, as in any engineering or design project, numerousimplementation-specific decisions must be made to achieve thedevelopers' specific goals, such as compliance with system-related andbusiness-related constraints, which may vary from one implementation toanother. Moreover, it should be appreciated that such a developmenteffort might be complex and time consuming, but would nevertheless be aroutine undertaking of design, fabrication, and manufacture for those ofordinary skill having the benefit of this disclosure.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” and “the” are intended to mean thatthere are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere may be additional elements other than the listed elements.Additionally, it should be understood that references to “oneembodiment” or “an embodiment” of the present disclosure are notintended to be interpreted as excluding the existence of additionalembodiments that also incorporate the recited features.

The present disclosure relates generally to handheld objects used forpointing and, more particularly, to determining a projected targetlocation of the handheld object. In particular, the reference elementmay provide an indication as to where a handheld object is pointing. Forexample, in the setting of a theme park, a user may point at an animatedobject (e.g., a robot or otherwise animated figure) of an attractionusing the handheld object, and, in response to detecting the location ofthe reference element, the animated object may output a user interactionexperience (e.g., wagging a tail). As another example, the user maypoint at a word on a poster, and, in response to detecting the locationof the reference element, a nearby speaker may output a voice speakingthe word. As yet another example, the user may point to an image of aperson on an electronic display, and, in response to detecting thelocation of the reference element, the display may play a video showingthe person in the image moving.

The presently disclosed systems and methods include using a camera todetermine locations of the reference element on a two-dimensional planeperpendicular to the direction of the camera. The camera may detect areference element of the handheld object, which may be made of amaterial (e.g., a retroreflective material) that is more easilydetectable by the camera. The location of the reference element may beused to determine a target location at which the user was pointing thehandheld object. However, in some systems, a user's perception as towhere they are pointing the handheld object may not match a projectedlocation of where the user is pointing that is determined based on thecamera's view. This could be due to a variety of factors, includingdominance of one eye over another (e.g., right eye-dominant or lefteye-dominant), tilting of the head, shifting of body weight, leaningtoward one side or another, and so on. Any combination of these factorsmay cause the user's perception of where they are aiming to shift, whiletheir hand is pointing the handheld object in the same location. Itshould be noted that a camera is an example of various light detectorsthat may be used in accordance with present embodiments. Accordingly,reference to a camera is representative of the other light detectorsthat may be used by embodiments of the present disclosure.

The presently disclosed systems and methods include providing acalibration point on the two-dimensional plane, at which a user maypoint the handheld object. The location of the reference element inrelation to the two-dimensional plane may be determined as an initiallocation, and one or more translation factors may be determined based onthe difference of the initial location and the calibration point. Thatis, the calibration point may correlate to where the user perceives theyare pointing the handheld object, while the initial location of thereference element may correlate to the location of the reference elementon the two-dimensional plane from the camera's point-of-view. Thedifference between the two may be used to translate subsequent detectedreference element locations on the two-dimensional plane from thecamera's point-of-view to projected target locations (e.g.,corresponding to where the user perceives they are pointing or intendsto point). That is, the one or more translation factors may compensatefor the difference between the user's perception as to where they arepointing the handheld object and the camera's determination of where thereference element is located on the two-dimensional plane.

Moreover, users move and point the handheld object using their arms,which may act as a radius of a sphere or spherical segment in a model ofthe interaction, with their shoulders being treated as a center of thesphere. As the users move the handheld object or points to differenttargets, the respective locations of the reference element of thehandheld object may vary between users despite pointing at the sametargets. This may be due to different arm lengths of the users.

Accordingly, the presently disclosed systems and methods determine aheight of the reference element (e.g., from the ground) based on theinitial location of the reference element, and estimate the user heightbased on the height of the reference element. From the user height, auser arm length may be estimated, which may be used to determine one ormore scaling factors. The one or more scaling factors may scale ormultiply the subsequent detected reference element locations on thetwo-dimensional plane from the camera's point-of-view to more accuratelydetermine the projected target locations (e.g., corresponding to wherethe user perceives they are pointing or intends to point). In thismanner, the one or more scaling factors may compensate for thedifference between user arm lengths.

Upon detection of a subsequent reference element location by the camera,the one or more translation factors and one or more scaling factors maybe applied to the subsequent reference element location to determine aprojected target location in relation to the two-dimensional plane.Present embodiments may include a processor that operates to analyzedata captured and communicated by the camera to provide relevant data,such as the translation factors, scaling factors, projected targetlocation in relation to the two-dimensional plane, and so forth.

Additionally, a user may move the handheld object in an arcuate orcircular nature due to their arm acting as a radius of a sphere orspherical segment, with their shoulder as a center. However, the camera,which determines the location of the reference element of the handheldobject on a flat two-dimensional plane, may distort a determinedlocation of the reference element due to the difference in shape betweenthe arcuate movement of the handheld object in space and the flattwo-dimensional plane detectable by the camera.

Accordingly, the presently disclosed systems and methods may determineone or more offsets to apply to the projected target location thatcompensate for this distortion. The one or more offsets may shift theprojected target location to increase or extend the distance between theprojected target location and the initial location in order tocompensate for the difference in shape between the arcuate nature of theuser's arm movement and the flat two-dimensional plane. For example, theone or more offsets may be determined using polynomial regression thatfits test data to one or more polynomial equations (e.g., polynomialequations of the third order).

In some embodiments, multiple reference element zones (e.g., where thereference element is located along an arc based on the user's arm) maybe determined that correspond to multiple projected target zones (e.g.,projected on the two-dimensional plane). Each projected target zone maycorrespond to a respective set of polynomial equations that mayaccurately compensate for the distortion applicable to that projectedtarget zone. As such, the camera may detect the reference element in areference element zone, a respective projected target zone may bedetermined that corresponds to the reference element zone, and arespective set of polynomial equations that corresponds to therespective projected target zone may be used to determine the one ormore offsets to be applied to the location of the reference element tocompensate for this distortion. In such embodiments, the multiplereference element zones may be different sizes (e.g., the referenceelement zones decrease in size the farther the reference element zone isfrom the two-dimensional plane) while the multiple projected targetzones are the same size, or the multiple reference element zones may bethe same size while the multiple projected target zones are differentsizes (e.g., the projected target zones increase in size the farther theprojected target zone is from the reference element).

By way of introduction, FIG. 1 is a diagram of a user 10 pointing ahandheld object 12 at a target 14, according to an embodiment of thepresent disclosure. The target 14 may be a physical object, a drawing, aphoto, a graphic, and so on. In some cases, the target 14 may be animage output by a display. The target 14 may be printed on, etched on,written on, projected on, attached on, or otherwise displayed on astructure 15. The user's perception is indicated by a first dashed line16. That is, the user 10 perceives that they are pointing the handheldobject 12 at the target 14, and specifically at a target location 17.However, due to certain human elements, such as dominance of one eyeover another, tilting of the head, shifting of body weight, leaningtoward one side or another, and so on, despite the user's perception orintention, the user 10 actually points the handheld object 12 at actualtarget location 18, as indicated by dashed line 19.

The handheld object 12 may be representative of or include any suitableobject the user 10 may use to point or refer to the target 14, such as astick, a pencil, a toy or model of a gun or weapon, a wand, and so on.The handheld object 12 may include a reference element 20, which mayfacilitate determining where the user 10 is pointing. In particular, acamera 22 may detect a location of the reference element 20, and thereference element 20 may be made of a material or device that enablesthe camera 22 to more easily detect the reference element 20. Forexample, the reference element 20 may be made of a retroreflectivematerial (e.g., retroreflective glass beads, microprisms, orencapsulated lenses sealed onto a fabric or plastic substrate), metaltape, and so on. In another example, the reference element 20 mayinclude an identifier (e.g., a unique graphical design, a barcode, aQuick Response (QR) code, and so on) that enables the camera 22 toidentify the reference element 20. As illustrated, the reference element20 may be located at an end 24 of the handheld object 12 opposite froman end 26 at which the user's hand 28 is holding the handheld object 12.This may facilitate determining the direction in which the user ispointing the handheld object 12, though the reference element 20 may bedisposed on any portion of the handheld object 12, or even the user 10.

The camera 22 may detect the location 30 of the reference element 20with respect to a two-dimensional plane 32. The location 30 may be usedto determine the target location 17 at which the user 10 perceives theyare pointing or intended to point by applying one or more translationfactors. As illustrated, the two-dimensional plane 32 may share the sameplane as the structure 15, though, in some embodiments, thetwo-dimensional plane 32 and the structure 15 may not share the sameplane. For example, the two-dimensional plane 32 and the structure 15may be parallel to one another. Moreover, to enable the camera 22 todetect the location 30 of the reference element 20, the structure 15 maybe made semi-transparent, transparent, or include any other suitableproperty that enables the camera 22 to detect the location 30 of thereference element 20.

In particular, one or more translation factors may be applied to thelocation 30 of the reference element 20 to compensate for a differencebetween the user's perception as to where they are pointing the handheldobject 12 and the camera's determination of where the reference element20 is located on the two-dimensional plane 32. The one or moretranslation factors may be determined during a calibration process wherethe user 10 points their handheld object 12 at a calibration point, andthe camera 22 detects this initial location of the reference element 20on the two-dimensional plane 32. The one or more translation factors mayrepresent one or more distances that the initial location is shifted toresult in the calibration point (e.g., with respect to thetwo-dimensional plane 32). Additionally, the one or more translationfactors may mitigate or compensate for dominance of one eye over another(e.g., right eye-dominant or left eye-dominant), tilting of the head,shifting of body weight, leaning toward one side or another, and so on.

Moreover, one or more scaling factors may be applied to the location 30of the reference element 20 to account or compensate for a differencebetween user arm lengths. That is, users move and point the handheldobject 12 using their arms, which may act as a radius of a sphere orspherical segment, with their shoulders as a center of the sphere. Asthe users move the handheld object 12 or point to different targets, therespective locations of the reference element 20 of the handheld object12 may vary between users despite pointing at the same targets, due todifferent arm lengths of the users.

Accordingly, a height of the reference element 20 (e.g., from theground) may be determined based on the initial location of the referenceelement 20, and the user height may be estimated based on the height ofthe reference element 20. From the user height, a user arm length may beestimated, which may be used to determine the one or more scalingfactors. The one or more scaling factors may scale or multiply thelocation 30 of the reference element 20 detected by the camera 22 on thetwo-dimensional plane 32.

Additionally, one or more offsets may be applied to the location 30 ofthe reference element 20 to generate the projected target location ofthe handheld object 12 to compensate for a distortion resulting from thearcuate or circular movement of the user's arm. That is, the distortionmay be caused by a difference in shapes between the arcuate movement andthe camera's detection of the location 30 of the reference element 20 onthe flat two-dimensional plane 32. The one or more offsets may shift theprojected target location to increase or extend the distance between theprojected target location and the initial location in order tocompensate for the difference in shape between the arcuate nature of theuser's arm movement and the flat two-dimensional plane. For example, theone or more offsets may be determined using polynomial regression thatfits test data to a polynomial equation, such as a third orderpolynomial equation.

In this manner, the projected target location of the handheld object 12may be generated, which may closely match the target location 17 atwhich the user 10 perceives that they are pointing the handheld object12. Advantageously, unlike certain other systems, only one point ofcalibration is used to determine the translation factors, the scalingfactors, and the offsets, and accurately determine the projected targetlocation of the handheld object 12. Whereas, in other applications(e.g., a pointing device used in presentations), it may not be asimportant to decrease calibration time as calibration may occur prior toan actual performance (e.g., during a preparation phase) and is notobserved by an audience or patron. However, in the instant case (e.g.,at an attraction of a theme park), it may be important to create animmersive user experience by hiding or preventing the user 10 fromnoticing that calibration is being performed. As such, reducing thecalibration process down to a single point (e.g., pointing the handheldobject 12 at a single calibration point) may serve to heighten orenhance the user experience.

With this in mind, FIG. 2 is a block diagram of a theme park attractionsystem 40, according to embodiments of the present disclosure. The themepark attraction system 40 may enable the user 10 to point the handheldobject 12 at various targets 14, and output a user interactionexperience based on determining the user 10 pointed the handheld object12 at a target 14. For example, the theme park attraction system 40 mayinclude a setting having characters popular with children, a televisionor movie-themed setting, a shooting gallery, a collection of targets,and so on.

The theme park attraction system 40 may include the handheld object 12with the reference element 20, as held and manipulated by the user 10.The theme park attraction system 40 may also include a user interactionsystem 42, which includes the camera 22 that detects a location of thereference element on the two-dimensional plane 32. The theme parkattraction system 40 may further include a projected locationdetermination system 44, which determines a projected target location ofthe handheld object 12. In particular, the projected target location mayrepresent a location on the two-dimensional plane 32 at which the user10 perceives they are pointing or intends to point. Indeed, the closerthe projected target location is to the target location 17, the moreaccurate the projected target location.

The projected location determination system 44 may include a controller46, having one or more processors (illustrated as a single processor 48)and one or more memory or storage devices (illustrated as a singlememory device 50). The processor 48 may execute software programs and/orinstructions stored in the memory device 50 that facilitate determiningthe projected target location of the handheld object 12. Moreover, theprocessor 48 may include multiple microprocessors, one or more“general-purpose” microprocessors, one or more special-purposemicroprocessors, and/or one or more application specific integratedcircuits (ASICS). For example, the processor 48 may include one or morereduced instruction set (RISC) processors. The memory device 50 maystore information such as control software, look up tables,configuration data, and so forth. The memory device 50 may include atangible, non-transitory, machine-readable-medium, such as volatilememory (e.g., a random access memory (RAM)), nonvolatile memory (e.g., aread-only memory (ROM)), flash memory, one or more hard drives, and/orany other suitable optical, magnetic, or solid-state storage medium. Thememory device 50 may store a variety of information and may be used forvarious purposes, such as instructions that facilitate the projectedtarget location of the handheld object 12.

The projected location determination system 44 may also includereference element location detection logic 52 that determines a locationof the reference element 20 on the two-dimensional plane 32. Inparticular, the projected location determination system 44 may becommunicatively coupled to the user interaction system 42 by anysuitable means, such as via wired communication or over a communicationnetwork using a wireless communication protocol or technology (e.g.,radio, Bluetooth, WiFi, infrared, Ethernet, Thread, ZigBee, Z-Wave, KNX,mobile, and/or microwave). The reference element location detectionlogic 52 may thus receive captured images (e.g., imagery) from thecamera 22 that show the reference element 20 on the two-dimensionalplane 32. The reference element location detection logic 52 maydetermine a location of the reference element 20 on the two-dimensionalplane 32 as expressed by, for example, a two-dimensional coordinate(e.g., x and y) system.

The projected location determination system 44 may further includetransformation logic 54 that transforms the location of the referenceelement 20, as determined by the reference element location detectionlogic 52, into a projected target location with respect to thetwo-dimensional plane 32. The transformation logic 54 includestranslation logic 56 that determines the one or more translation factorsthat compensate for a difference between the user's perception as towhere they are pointing the handheld object 12 and the camera'sdetermination of where the reference element 20 is located on thetwo-dimensional plane 32.

In particular, the translation logic 56 may determine the one or moretranslation factors by performing a single-point calibration process.This process includes receiving a calibration location on thetwo-dimensional plane 32, receiving a location of the reference element20 on the two-dimensional plane 32 (e.g., corresponding to when the user10 points the handheld object 12 at the calibration location), anddetermining the one or more translation factors based on the differencein locations between the calibration location and the location of thereference element 20.

FIG. 3 is a diagram of a user 10 pointing a handheld object 12 at thecalibration location 80, according to an embodiment of the presentdisclosure. The calibration location 80 may correspond to a physicalobject, a drawing, a photo, a graphic, and so on. In some cases, thecalibration location 80 may correspond to an image output by a display.The user 10 may be prompted by instructions provided in any suitableformat (e.g., written, etched, printed, attached, or displayed on thestructure 15). The calibration location 80 may be provided to enableusers to position their arms similarly to enable a controlled manner todetect user height, while also enabling the projected locationdetermination system 44 of FIG. 2 to determine a difference between theuser's perception as to where they are pointing the handheld object 12and where the user 10 is actually pointing the handheld object 12. Forexample, the calibration location 80 may be located to enable the user10 to extend their arm 82 as close to parallel as possible to the ground84, at a certain angle with respect to a plane parallel to the ground,and so on. In some embodiments, the calibration location 80 may becustomized for user heights. That is, in some embodiments, thecalibration location 80 may be located lower on the structure 15 forusers sitting in vehicles, such as wheelchairs, personal electricvehicles, strollers, and so on. As another example, the calibrationlocation 80 may be located higher on the structure 15 for adults thanfor children, the calibration location 80 may be located higher on thestructure 15 for male users than for female users, and so on.

As such, the calibration location 80 may be predetermined and known bythe projected location determination system 44. Upon prompting, the user10 may extend their arm 82 and point the handheld object 12 at thecalibration location 80. However, due to the distortion effects causedby the human body, such as dominance of one eye over another, tilting ofthe head, shifting of body weight, leaning toward one side or another,the user's choice of hand holding the handheld object 12 (e.g., righthand vs. left hand), physical limitations (e.g., that affect range ofmotion), whether the user's movement may be altered due to anencumbrance (e.g., a backpack or holding a child) and so on, despite theuser's perception of or intent to point the handheld object 12 at thecalibration location 80 as indicated by the dashed line 85, the user 10may actually point the handheld object 12 at another location, such asactual calibration location 86, as indicated by dashed line 88.

The camera 22 detects the location 90 of the reference element 20 on thetwo-dimensional plane 32, and sends an indication of the location 90 tothe projected location determination system 44. The translation logic56, which may be part of a model of human interactions, may thendetermine a difference in location between the location 90 of thereference element 20 and the predetermined calibration location 80,which may be expressed in two-dimensional (e.g., x and y) coordinates.The translation logic 56 may use the difference to generate one or moretranslation factors that may be applied to subsequent detected locationsof the reference element 20 to shift the subsequent detected locationsof the reference element 20 and determine subsequent projected targetlocations of the handheld object 12 that correspond to where the user 10intended to point the handheld object 12. The translation factors may beprovided in the form of a transformation matrix, which may be applied toa subsequent detected location of the reference element 20 to generate aprojected target location of the reference element 20, as shown below:

$\begin{matrix}{{{❘\begin{matrix}x \\y \\1\end{matrix}❘}{❘\begin{matrix}1 & 0 & X \\0 & 1 & Y \\0 & 0 & 1\end{matrix}❘}} = {❘\begin{matrix}x^{\prime} \\y^{\prime} \\1\end{matrix}❘}} & {{Equation}1}\end{matrix}$

where: x=the horizontal component of the location 90 of the referenceelement 20 on the two-dimensional plane 32;

-   -   y=the vertical component of the location 90 of the reference        element 20 on the two-dimensional plane 32;    -   X=the horizontal difference between the reference element 20 and        the calibration location 80 on the two-dimensional plane 32;    -   Y=the vertical difference between the reference element 20 and        the calibration location 80 on the two-dimensional plane 32;    -   x′=the horizontal component of the projected target location of        the handheld object 12 on the two-dimensional plane 32; and    -   y′=the vertical component of the projected target location of        the handheld object 12 on the two-dimensional plane 32.

For example, FIG. 4 is a diagram of an example of applying the one ormore translation factors to a subsequent detected location 120 of thereference element 20, according to embodiments of the presentdisclosure. As illustrated, during calibration, the location 90 of thereference element 20 is 4 units (e.g., centimeters) to the right of thecalibration location 80 and 2 units (e.g., centimeter) up from thecalibration location 80. As such, the translation factors may include +4in the horizontal direction and +2 in the vertical direction.Accordingly, X may be set to +4 and Y may be set to +2 in thetransformation matrix. The translation logic 56 may apply thetransformation matrix to the subsequent detected location 120 (e.g., [2,1]) of the reference element 20 to shift the subsequent detectedlocation 120 to the right by 4 units and up 2 units, to generate theprojected target location 122 at 6 units to the right of the calibrationlocation 80 and 3 units up (e.g., [6, 3]). Thus, the translation logic56 may compensate for the difference between the user's perception as towhere they are pointing the handheld object 12 and the camera'sdetermination of where the reference element 20 is located on thetwo-dimensional plane 32.

Turning back to FIG. 2, the transformation logic 54 may also includescaling logic 58 that determines the one or more scaling factors thatcompensate for differences between user arm lengths. That is, as shownin FIG. 3, users 10 move and point the handheld object 12 using theirarms 82, which may act as a radius of a sphere or spherical segment 92,with their shoulders as a center 94 of the sphere. As the users 10 movethe handheld object 12 to point to different targets, the respectivelocations of the reference element 20 of the handheld object 12 may varybetween users 10 despite pointing at the same targets, due to differentarm lengths of the users 10.

In particular, the scaling logic 58 may determine the one or morescaling factors based on the location 90 of the reference element 20detected by the camera 22 during the calibration process. The height 96of the camera 22 from the ground 84 may be predetermined and known bythe scaling logic 58. Thus, the scaling logic 58 may determine a height98 of the reference element 20 from the ground 84 based on the location90 of the reference element 20 and the predetermined height 96. Based onthe height 98 of the reference element 20, user height estimation logic60 of the scaling logic 58 may determine the user's height 100. Inparticular, test or sample data may be collected of the locations 90 ofthe reference element 20 when users 10 point the handheld object 12 atthe calibration location 80 and the heights of those users 10. Theheights 102 of the locations 90 of the reference element 20 may becorrelated to the heights of the users 10, and the scaling logic 58 mayestimate the user's height 100 based on this predetermined correlationand the height 98 of the reference element 20. The model for identifyingcorrelations may be populated with tables of standard correlationbetween height and reach (e.g., a ratio between height and arm lengthfor various body types in a population).

User arm length estimation logic 62 of the scaling logic 58 may thenestimate the user's arm length 104 based on the user height 100. Theestimation may be made based on a predetermined correlation (e.g., analgorithm or table based on empirical data) between arm length 104 anduser height 100. This predetermined correlation may be determined basedon test or sample data, scientific data associated with proportions ofthe human body, and/or any other suitable source.

The scaling logic 58 may determine the one or more scaling factors basedon the user's arm length 104. For example, when pointing away from aninitial location (e.g., the calibration location 80), the camera 22 maydetect the location of the reference element 20 to be closer to theinitial location with a user 10 having a shorter arm length 104 comparedto a user 10 having a longer arm length. As such, the scaling logic 58may determine larger scaling factors for users 10 having longer armlengths 104 compared to users 10 having shorter arm lengths 104. Thescaling logic 58 may apply the one or more scaling factors to asubsequent detected location of the reference element 20 to scale (e.g.,diminish or expand) the location to generate a projected target locationof the reference element 20. The scaling factors may include horizontaland vertical components, be provided in the form of a transformationmatrix, and inserted into the transformation matrix that includestranslation factors from Equation 1 above, as shown below:

$\begin{matrix}{{{❘\begin{matrix}x \\y \\1\end{matrix}❘}{❘\begin{matrix}{k_{1} \star Y} & 0 & X \\0 & {k_{2} \star Y} & Y \\0 & 0 & 1\end{matrix}❘}} = {❘\begin{matrix}x^{\prime} \\y^{\prime} \\1\end{matrix}❘}} & {{Equation}2}\end{matrix}$

where: k₁=a horizontal scaling factor generated based on user arm length104; and

k₂=a vertical scaling factor generated based on user arm length 104.

The values of the scaling factors k₁ and k₂ may be determined based oncorrelating test or sample data collected from users 10 pointing thehandheld object 12 at various targets and the arm lengths 104 of thoseusers 10. For example, the scaling logic 58 may determine that theheight 98 of the reference element 20 from the ground 84 is 1.25 metersbased on image data (e.g., a first or calibration image of imagery)received from the camera 22. The user height estimation logic 60 maydetermine that the user's height 100 is approximately 1.8 meters basedon the height 98 of the reference element 20. The user arm lengthestimation logic 62 may determine that the user's arm length 104 is 0.6meters based on the user's height 100. The scaling logic 58 may thendetermine that the horizontal scaling factor k₁ is 1.5 and the verticalscaling factor k₂ is 1.75 based on the user's arm length 104.Accordingly, the scaling logic 58 may generate the transformation matrixin Equation 2 with k₁=1.5 and k₂=1.75, and the projected locationdetermination system 44 may apply the transformation matrix to asubsequent detected location of the reference element 20 to generate aprojected target location of where the user 10 intended to point thehandheld object 12, that compensates for differences in user arm length104.

For example, FIG. 5 is a diagram of an example of applying scalingfactors to a subsequent detected location 120 of the reference element20, according to embodiments of the present disclosure. As illustrated,the subsequent detected location 120 of the reference element 20 is 4units (e.g., centimeters) to the right of the calibration location 80and 4 units (e.g., centimeters) up from the calibration location 80(e.g., [4, 4]). Applying the transformation matrix of Equation 2 havingthe horizontal scaling factor k₁=1.5 and the vertical scaling factork₂=1.75 to the subsequent detected location 120 results in scaling thesubsequent detected location 120 horizontally by 1.5, thus generating aprojected target location 130 6 units to the right of the calibrationlocation 80, and vertically by 1.7, thus generating a projected targetlocation 130 7 units (e.g., centimeters) up (e.g., [6, 7]). Thus, thescaling logic 58 may compensate for differences in user arm lengths 104.

Turning back to FIG. 2, the projected location determination system 44may include arc distortion compensation logic 64 that compensates forthe difference in shape between the arcuate nature 92 of the user's armmovement and the flat two-dimensional plane 32. For example, FIG. 6 is adiagram of a user 10 pointing the handheld object 12 at differenttargets. As illustrated, an angle θ formed between a first position 140of the user's arm 82 and a second position 142 of the user's arm 82 isthe same as between a third position 144 of the user's arm 82 and afourth position 146 of the user's arm 82. However, as viewed andcaptured by the camera 22 on the two-dimensional plane 32, a distance hobetween a first reference element location 148 corresponding to thefirst position 140 of the user's arm 82 and a second reference elementlocation 150 corresponding to the second position 142 of the user's arm82 is different (e.g., greater than) a distance hi between a thirdreference element location 152 corresponding to the third position 144of the user's arm 82 and a fourth reference element location 154corresponding to the fourth position 146 of the user's arm 82.

As such, the arc distortion compensation logic 64 may determine one ormore offsets to apply to the projected target location that compensatesfor this distortion. The one or more offsets may shift the projectedtarget location to increase or extend the distance between the projectedtarget location and an initial location (e.g., the calibration location80) to compensate for the difference in shape between the arcuate nature92 of the user's arm movement and the flat two-dimensional plane 32. Forexample, the one or more offsets may be determined using regressionanalysis that fits test or sample data from users 10 pointing thehandheld object 12 at various targets (e.g., with the reference element20 along the arc 92) to an equation. In some embodiments, the arcdistortion compensation logic 64 may fit the test data to a polynomialequation (e.g., a polynomial equation of the third order), though anysuitable order or type of equation may be used. For example, a firstpolynomial equation of the third order (Equations 3 and 4 below) may beused to determine a horizontal offset to be applied to the projectedtarget location that compensates for this distortion in the horizontaldirection, and a second polynomial equation of the third order(Equations 5 and 6 below) may be used to determine a vertical offset tobe applied to the projected target location that compensates for thedistortion in the vertical direction:

x _(offset)=Σ_(i=0) ³Σ_(j=0) ³ a _(ij) x ^(i) y ^(j)  Equation 3

(which may be additionally or alternatively represented as:

x _(offset) =ax ³ +by ³ +cx ² y+dxy ² +ex ² +fy ² +gxy+hx+ky+l)  Equation 4

y _(offset)=Σ_(i=0) ³Σ_(j=0) ³ b _(ij) x ^(i) y ^(j)  Equation 5

(which may be additionally or alternatively represented as:

y _(offset) =ay ³ +bx ³ +cy ² x+dyx ² +ey+fx ² +gyx+hy+kx+l   Equation 6

where: x_(offset)=the horizontal offset to be applied to a projectedtarget location;

-   -   y_(offset)=the vertical offset to be applied to the projected        target location;    -   x=the horizontal component of the projected target location;    -   y=the vertical component of the projected target location; and    -   a_(i), b_(i), c_(i), a, b, c, d, e, f, g, h, k, and l=constants        that are determined using regression analysis, wherein each        constant may be different from Equation to Equation (e.g.,        constant a in Equation 4 may be different from constant a in        Equation 6).

The horizontal component of the projected target location may bemeasured as a horizontal distance away from an initial location (e.g.,corresponding to the calibration location 80 and/or when the user 10points the handheld object 12 directly at the camera 22), while thevertical component of the projected target location may be measured as avertical distance away from the initial location. As previouslymentioned, for any of the polynomial Equations 3-6, the constants a_(i),b_(i), c_(i), d, e, f, g, h, k, and l may be determined by fitting testor sample data to a polynomial equation using polynomial regressionanalysis (and may be different between the Equations). As such, the oneor more offsets may be determined for each projected target location asthe user 10 moves and points the handheld object 12.

However, applying any of Equations 3-6 to determine the horizontal andvertical offsets for each projected target location as the user 10 movesand points the handheld object 12 may be time-consuming and useexcessive computing resources (e.g., processing, memory, storage, ornetworking resources). As such, to more efficiently compensate for thedifference in shape between the arcuate nature 92 of the user's armmovement and the flat two-dimensional plane 32, in some embodiments, thearc distortion compensation logic 64 may divide the arc 92 at which thereference element 20 may be located into multiple reference elementzones, each of which may correspond to a respective projected targetzone (e.g., projected on the two-dimensional plane). Each projectedtarget zone may correspond to a respective set of polynomial equationsthat may accurately compensate for the distortion applicable to thatprojected target zone. As such, the camera 22 may detect the referenceelement 20 in a reference element zone, the arc distortion compensationlogic 64 may determine a respective projected target zone thatcorresponds to the reference element zone, and the arc distortioncompensation logic 64 may apply a respective set of polynomial equationsthat corresponds to the respective projected target zone to the locationof the reference element to determine the one or more offsets to beapplied to the location of the reference element to compensate for thisdistortion. In such embodiments, the multiple reference element zonesmay be different sizes (e.g., the reference element zones decrease insize the farther the reference element zone is from the two-dimensionalplane 32) while the multiple projected target zones are the same size,or the multiple reference element zones may be the same size while themultiple projected target zones are different sizes (e.g., the projectedtarget zones increase in size the farther the projected target zone isfrom the reference element 20).

FIG. 7 is a diagram of differently-sized multiple reference elementzones 170 and uniformly-sized multiple projected target zones 172,according to embodiments of the present disclosure. As illustrated, afirst reference element zone 174 closest to the two-dimensional plane 32is largest in size, a second reference element zone 176 next closest tothe two-dimensional plane 32 is next largest in size (but smaller thanthe first reference element zone 174), a third reference element zone178 next closest to the two-dimensional plane 32 is next largest in size(but smaller than the second reference element zone 176), and a fourthreference element zone 180 next closest to the two-dimensional plane 32is next largest in size (but smaller than the third reference elementzone 178). While four reference element zones 170 are illustrated inFIG. 7, it should be understood that any suitable number of referenceelement zones 170 are contemplated of any suitable size, where thereference element zones 170 decrease in size the farther the referenceelement zone 170 is from the two-dimensional plane 32. Moreover, eachprojected target zone 172 is the same size as other projected targetzones 172, corresponds to a respective reference element zone 170, andcorresponds to a respective set of polynomial equations that generaterespective offsets (e.g., horizontal and vertical offsets) that may beapplied to a location of the reference element 20. In particular, eachset of polynomial equations that corresponds to a respective projectedtarget zone 172 may have different value sets for constants a_(i),b_(i), c_(i), a, b, c, d, e, f, g, h, k, and l, as provided in any ofEquations 3-6 (and may be different between the Equations). Decreasingthe sizes of the reference element zones 170 the farther the referenceelement zone 170 is from the two-dimensional plane 32, while maintainingthe same sizes of the projected target zones 172, may enable the arcdistortion compensation logic 64 to compensate for the difference inshape between the arcuate nature 92 of the user's arm movement and theflat two-dimensional plane 32 in an efficient and resource-conservingmanner.

FIG. 8 is a diagram of uniformly-sized multiple reference element zones190 and differently-sized multiple projected target zones 192, accordingto embodiments of the present disclosure. As illustrated, each referenceelement zone 190 is the same size. However, a first projected targetzone 194 closest to the reference element 20 is smallest in size, asecond projected target zone 196 next closest to the reference element20 is next smallest in size (but larger than the first projected targetzone 194), a third projected target zone 198 next closest to thereference element 20 is next smallest in size (but larger than thesecond projected target zone 196), and a fourth projected target zone200 next closest to the reference element 20 is next smallest in size(but larger than the third projected target zone 198). While fourprojected target zones 192 are illustrated in FIG. 8, it should beunderstood that any suitable number of projected target zones 192 arecontemplated of any suitable size, where the projected target zones 192increase in size the farther the projected target zone 192 is from thereference element 20. Each projected target zone 192 corresponds to arespective reference element zone 190, and also corresponds to arespective set of polynomial equations that generate respective offsets(e.g., horizontal and vertical offsets) that may be applied to alocation of the reference element 20. In particular, each set ofpolynomial equations that corresponds to a respective projected targetzone 192 may have different value sets for constants a_(i), b_(i) c_(i),a, b, c, d, e, f, g, h, j, k, and l, as provided in any of Equations 3-6(and may be different between the Equations). Increasing the sizes ofthe projected target zones 192 the farther the projected target zone 192is from the reference element 20, while maintaining the same sizes ofthe reference element zones 190, may enable the arc distortioncompensation logic 64 to compensate for the difference in shape betweenthe arcuate nature 92 of the user's arm movement and the flattwo-dimensional plane 32 in an efficient and resource-conserving manner.

It should be noted that, for the purpose of simplicity, FIGS. 6-8illustrate the distortion caused by the difference in shape between thearcuate nature 92 of the user's arm movement and the flattwo-dimensional plane 32 in only the vertical (e.g., y) direction.However, the presently disclosed systems and methods contemplatecompensating for the distortion in any suitable direction, including thehorizontal (e.g., x) direction, as evidenced by Equations 3 and 4, whichprovide a horizontal offset to compensate for distortion in thehorizontal direction, and the vertical (e.g., y) direction, as evidencedby Equations 5 and 6, which provide a vertical offset to compensate fordistortion in the vertical direction.

Turning back to FIG. 2, if the projected location determination system44 determines that the projected target location corresponds to a target14 printed on, etched on, written on, attached on, or otherwisedisplayed on the structure 15. Then an output device 66 of the userinteraction system 42 may output a user interaction experience. Theoutput device 66 may be any suitable device that is capable ofoutputting a desired user interaction experience, such as an electronicdisplay, a speaker, a virtual reality device, an augmented realitydevice, an actuator, and/or an animated device (e.g., a robotic figure).The target 14 may be a part of, fixed to, attached to, or include theoutput device 66, or the target 14 may be separate from the outputdevice 66. For example, in the setting of a theme park, the target 14and the output device 66 may both be an animated object of anattraction, and, in response to determining that the projected targetlocation corresponds to the animated object, the animated object mayoutput a user interaction experience (e.g., wagging a tail). As anotherexample, the target 14 may be a word printed on a poster and the outputdevice 66 may be a nearby speaker, and, in response to determining thatthe projected target location corresponds to the word printed on theposter, the nearby speaker may output a voice speaking the word. As yetanother example, the target 14 may be an image of a person on anelectronic display and the output device 66 may be the electronicdisplay, and, in response to determining that the projected targetlocation corresponds to the image of the person, the electronic displaymay play a video showing the person of the image performing a signatureaction.

With this in mind, FIG. 9 is a flow diagram of a process 210 fordetermining a projected target location of a handheld object 12,according to embodiments of the present disclosure. The process 210 maybe performed by any suitable device that may determine the projectedtarget location of the handheld object 12, such as any component of theprojected location determination system 44, including the controller 46,the processor 48, the reference element location detection logic 52, thetransformation logic 54, the translation logic 56, the scaling logic 58,the user height estimation logic 60, and/or the user arm length logic62. While the process 210 is described using steps in a specificsequence, it should be understood that the present disclosurecontemplates that the described steps may be performed in differentsequences than the sequence illustrated, and certain described steps maybe skipped or not performed altogether. In some embodiments, the process210 may be implemented by executing instructions stored in a tangible,non-transitory, computer-readable medium, such as the memory device 50,using a processor, such as the processor 48.

As illustrated, in process block 212, the processor 48 receives anindication to calibrate a handheld object 12. The indication may be inthe form of an image (e.g., a first or calibration image of imagery)captured by the camera 22, which includes a presence of the referenceelement 20 of the handheld object 12. In some embodiments, a motionsensor or other suitable sensor capable of indicating that a user 10 hasentered a viewing area of the camera 22 with a handheld object 12 havingthe reference element 20 may provide the indication.

In process block 214, the processor 48 receives a calibration location80. In particular, the calibration location 80 may be predetermined andknown to the processor 48, as the calibration location 80 may be fixedon the structure 15 or displayed by the processor 48 on the structure15.

In process block 216, the processor 48 receives a location of thereference element 20 of the handheld object 12. For example, the camera22 may provide an image (e.g., a second or subsequent image of imagerycaptured by the camera 22) of the reference element 20. The processor 48may then instruct the reference element location detection logic 52 todetermine the location of the reference element 20 on thetwo-dimensional plane 32.

In process block 218, the processor 48 instructs the translation logic56 to determine one or more translation factors based on the location ofthe reference element 20 and the calibration location 80. The one ormore translation factors may compensate for a difference between theuser's perception as to where they are pointing the handheld object 12and the camera's determination of where the reference element 20 islocated on the two-dimensional plane 32. In particular, the translationlogic 56 may determine the one or more translation factors by performinga single-point calibration process. This process includes receiving acalibration location on the two-dimensional plane 32, receiving alocation of the reference element 20 on the two-dimensional plane 32(e.g., corresponding to when the user 10 points the handheld object 12at the calibration location), and determining the one or moretranslation factors based on the difference in locations between thecalibration location and the location of the reference element 20.

The translation logic 56 may use the difference to generate the one ormore translation factors that may be applied to subsequent detectedlocations of the reference element 20 to shift the subsequent detectedlocations of the reference element 20 and determine subsequent projectedtarget locations of the handheld object 12 that correspond to where theuser 10 intended to point the handheld object 12. The translationfactors may be provided in the form of a transformation matrix, whichmay be applied to a subsequent detected location of the referenceelement 20 to generate a projected target location of the referenceelement 20, as shown in Equation 1.

In process block 220, the processor 48 instructs the user heightestimation logic 60 to determine a height 100 of the user 10 based onthe location of the reference element 20. In process block 222, theprocessor 48 instructs the user arm length estimation logic 62 todetermine the arm length 104 of the user 10 based on the height 100 ofthe user 10.

In process block 224, the processor 48 instructs the scaling logic 58 todetermine one or more scaling factors based on the arm length 104 of theuser 10. The scaling logic 58 may provide the scaling factors in thetransformation matrix of Equation 2 as shown above. The scaling factorsmay compensate for differences in user arm length 104 by scaling (e.g.,multiplying) on the location of the reference element 20 with respect toan initial location (e.g., the calibration location 80).

In process block 226, the processor 48 instructs the transformationlogic 54 to determine a projected target location of the handheld object12 based on the location of the reference element 20, the one or moretranslation factors, and the one or more scaling factors. In particular,the transformation logic 54 may apply the transformation matrix ofEquation 2 that includes the one or more translation factors and the oneor more scaling factors to the location of the reference element 20 togenerate the projected target location. That is, the projected targetlocation may correspond to where the user 10 perceives they are pointingor intends to point.

In decision block 228, the processor 48 determines whether the projectedtarget location correlates with a user interaction element. The userinteraction element may be any suitable target that serves as a triggerto perform a user interaction experience. For example, the userinteraction element may include any feature of interest that the user 10may expect, when pointing at with the handheld object 12, would causethe user interaction experience to be performed.

If the processor 48 determines that the projected target locationcorrelates with a user interaction element, then, in process block 230,the processor 48 instructs the user interaction system 42 to perform arespective user interaction experience using the appropriate outputdevice 66. For example, the output device 66 may be an animated objectof an attraction, and the user interaction system 42 may cause theanimated object to bark, meow, speak, move, blink, and so on. As anotherexample, the output device 66 may be a speaker, and the user interactionsystem 42 may cause the speaker to output a sound, voice, music, and soon. As yet another example, the output device 66 may be an electronicdisplay, and the user interaction system 42 may cause the electronicdisplay to display an image, play a video, and so on.

If the processor 48 determines that the projected target location doesnot correlate with a user interaction element, then, in decision block232, the processor 48 determines whether a next location of thereference element 20 has been received. If so, the processor 48 repeatsprocess block 226 and determines the projected target location of thehandheld object 12 based on the next location of the reference element20 and the translation factors and scaling factors that have alreadybeen determined from process blocks 218 and 224.

If the processor 48 determines that a next location of the referenceelement 20 has not been received, then the processor 48 repeats processblock 212 to receive a next indication to calibrate the handheld object12 (e.g., from a next user 10). In this manner, the process 210 maydetermine a projected target location of the handheld object 12 usingsingle-point calibration (e.g., without requiring the user 10 to pointthe handheld object 12 at more than one point to calibrate the projectedlocation determination system 44) that compensates for both a differencebetween the user's perception as to where they are pointing the handheldobject 12 and the camera's determination of where the reference element20 is located on the two-dimensional plane 32, as well as differences inuser arm length 104.

Moreover, the projected location determination system 44 may alsocompensate for the distortion caused by the difference in shape betweenthe arcuate nature 92 of the user's arm movement and the flattwo-dimensional plane 32, as illustrated in FIG. 6. FIG. 10 is a flowdiagram of a process 240 for compensating for this distortion, accordingto embodiments of the present disclosure. The process 240 may beperformed by any suitable device that may compensate for thisdistortion, such as any component of the projected locationdetermination system 44, including the controller 46, the processor 48,and/or the arc distortion compensation logic 64. While the process 240is described using steps in a specific sequence, it should be understoodthat the present disclosure contemplates that the described steps may beperformed in different sequences than the sequence illustrated, andcertain described steps may be skipped or not performed altogether. Insome embodiments, the process 240 may be implemented by executinginstructions stored in a tangible, non-transitory, computer-readablemedium, such as the memory device 50, using a processor, such as theprocessor 48.

As illustrated, in process block 242, the processor 48 receives alocation of a reference element 20 of a handheld object 12. In someembodiments, the processor 48 may receive a projected target location ofthe handheld object 12.

In process block 244, the processor 48 determines a horizontal offsetbased on the location of the reference element 20 and a first polynomialequation. In particular, the processor 48 may receive the projectedtarget location of the handheld object 12, or determine the projectedtarget location using the process 210 of FIG. 9. The processor 48 maythen instruct the arc distortion compensation logic 64 to applypolynomial Equation 3 or 4 to the projected target location of thehandheld object 12 to determine the horizontal offset.

In process block 246, the processor 48 determines a vertical offsetbased on the location of the reference element 20 and a secondpolynomial equation. In particular, the processor 48 may instruct thearc distortion compensation logic 64 to apply polynomial Equation 5 or 6to the projected target location of the handheld object 12 to determinethe vertical offset.

In process block 248, the processor 48 determines a projected targetlocation of the handheld object 12 based on the location of thereference element 20, the horizontal offset, and the vertical offset. Inparticular, the processor 48 may instruct the arc distortioncompensation logic 64 to apply (e.g., add) the horizontal offset to ahorizontal component (e.g., the x-coordinate) of the projected targetlocation and apply (e.g., add) the vertical offset to a verticalcomponent (e.g., the y-coordinate) of the projected target location togenerate the projected target location.

In some embodiments, to more efficiently compensate for the differencein shape between the arcuate nature 92 of the user's arm movement andthe flat two-dimensional plane 32, the arc distortion compensation logic64 may divide the arc 92 at which the reference element 20 may belocated into multiple reference element zones, each of which maycorrespond to a respective projected target zone (e.g., projected on thetwo-dimensional plane). Each projected target zone may correspond to arespective set of polynomial equations that may accurately compensatefor the distortion applicable to that projected target zone. As such,the camera 22 may detect the reference element 20 in a reference elementzone, the arc distortion compensation logic 64 may determine arespective projected target zone that corresponds to the referenceelement zone, and the arc distortion compensation logic 64 may apply arespective set of polynomial equations that corresponds to therespective projected target zone to the location of the referenceelement to determine the one or more offsets to be applied to thelocation of the reference element to compensate for this distortion. Insuch embodiments, the multiple reference element zones may be differentsizes (e.g., the reference element zones decrease in size the fartherthe reference element zone is from the two-dimensional plane 32) whilethe multiple projected target zones are the same size, as shown in FIG.7, or the multiple reference element zones may be the same size whilethe multiple projected target zones are different sizes (e.g., theprojected target zones increase in size the farther the projected targetzone is from the reference element 20), as shown in FIG. 8.

In this manner, the process 240 may compensate for the arcuate nature 92of the user's arm movement and the flat two-dimensional plane 32.Moreover, to compensate for the difference between the user's perceptionas to where they are pointing the handheld object 12 and the camera'sdetermination of where the reference element 20 is located on thetwo-dimensional plane 32, differences in user arm length 104, and thedifference in shape between the arcuate nature 92 of the user's armmovement and the flat two-dimensional plane 32, the process 240 of FIG.10 may be performed before, after, or as part of the process 210 of FIG.9.

While the embodiments set forth in the present disclosure may besusceptible to various modifications and alternative forms, specificembodiments have been shown by way of example in the drawings and havebeen described in detail herein. However, it should be understood thatthe disclosure is not intended to be limited to the particular formsdisclosed. The disclosure is to cover all modifications, equivalents,and alternatives falling within the spirit and scope of the disclosureas defined by the following appended claims.

The techniques presented and claimed herein are referenced and appliedto material objects and concrete examples of a practical nature thatdemonstrably improve the present technical field and, as such, are notabstract, intangible or purely theoretical. Further, if any claimsappended to the end of this specification contain one or more elementsdesignated as “means for [perform]ing [a function] . . . ” or “step for[perform]ing [a function] . . . ”, it is intended that such elements areto be interpreted under 35 U.S.C. 112(f). However, for any claimscontaining elements designated in any other manner, it is intended thatsuch elements are not to be interpreted under 35 U.S.C. § 112(f).

1. A method, comprising: determining a first offset along a first axisof a two-dimensional plane that compensates for distortion along thefirst axis associated with an arc of movement of a reference element ofa handheld object in a three-dimensional space by dividing the arc intoa plurality of reference element zones, and wherein each referenceelement zone of the plurality of reference element zones is associatedwith a respective target zone of a plurality of target zones on thetwo-dimensional plane; determining a projected target location of thehandheld object based on an initial location of the reference element ofthe handheld object on the two-dimensional plane and the first offset;and outputting a user interactive experience in response to determiningthat the projected target location corresponds to a target location. 2.The method of claim 1, comprising receiving the initial location of thereference element of the handheld object as projected on thetwo-dimensional plane.
 3. The method of claim 2, comprising: determininga second offset along a second axis of the two-dimensional plane thatcompensates for distortion along the second axis associated with the arcin the three-dimensional space by dividing the arc into a plurality ofadditional reference element zones, and wherein each additionalreference element zone of the plurality of additional reference elementzones is associated with an additional respective target zone of aplurality of additional target zones on the two-dimensional plane; anddetermining the projected target location of the handheld object basedon the initial location of the reference element of the handheld object,the first offset, and the second offset.
 4. The method of claim 3,wherein each additional reference element zone of the plurality ofadditional reference element zones is a same size, and wherein eachadditional target zone of the plurality of additional target zones is adifferent size.
 5. The method of claim 3, wherein a first additionaltarget zone of the plurality of additional target zones is farther awayfrom the reference element than a second additional target zone of theplurality of additional target zones, and wherein the first additionaltarget zone is larger in size than the second additional target zone. 6.The method of claim 3, wherein each additional reference element zone ofthe plurality of additional reference element zones is a different size,and wherein each additional target zone of the plurality of additionaltarget zones is a same size.
 7. The method of claim 3, wherein a firstadditional reference element zone of the plurality of additionalreference element zones is farther from the two-dimensional plane than asecond additional reference element zone of the plurality of additionalreference element zones, and wherein the first additional referenceelement zone is smaller in size than the second additional referenceelement zone.
 8. The method of claim 3, wherein determining theprojected target location comprises shifting the initial location of thereference element by the first offset along the first axis, the secondoffset along the second axis, or both.
 9. The method of claim 3, whereinthe first offset along the first axis comprises a horizontal offset onthe two-dimensional plane and the second offset along the second axiscomprises a vertical offset on the two-dimensional plane.
 10. The methodof claim 3, wherein determining the first offset comprises performingregression analysis to fit test data to a first equation, and whereindetermining the second offset comprises performing additional regressionanalysis to fit the test data to a second equation.
 11. The method ofclaim 10, wherein the first equation and the second equation are eachpolynomial equations of a third order.
 12. The method of claim 11,wherein the first polynomial equation compensates for the distortionalong the first axis associated with the arc and the respective targetzone on the two-dimensional plane, and wherein the second polynomialequation compensates for the distortion along the second axis associatedwith the arc and the respective additional target zone on thetwo-dimensional plane.
 13. The method of claim 1, wherein each referenceelement zone of the plurality of reference element zones is a same size,and wherein each target zone of the plurality of target zones is adifferent size.
 14. The method of claim 1, wherein a first target zoneof the plurality of target zones is farther away from the referenceelement than a second target zone of the plurality of target zones, andwherein the first target zone is larger in size than the second targetzone.
 15. The method of claim 1, wherein each reference element zone ofthe plurality of reference element zones is a different size, andwherein each target zone of the plurality of target zones is a samesize.
 16. The method of claim 1, wherein a first reference element zoneof the plurality of reference element zones is farther from thetwo-dimensional plane than a second reference element zone of theplurality of reference element zones, and wherein the first referenceelement zone is smaller in size than the second reference element zone.17. A system comprising: an output device configured to output a userinteractive experience; a controller having one or more processors and amemory, the memory storing machine-readable instructions configured tocause the one or more processors to: determine a first offset along afirst axis of a two-dimensional plane that compensates for distortionalong the first axis associated with an arc of movement of a referenceelement of a handheld object in a three-dimensional space by dividingthe arc into a plurality of reference element zones, and wherein eachreference element zone of the plurality of reference element zones isassociated with a respective target zone of a plurality of target zoneson the two-dimensional plane; determine a projected target location ofthe handheld object based on an initial location of the referenceelement of the handheld object on the two-dimensional plane and thefirst offset; and cause the output device to output the user interactiveexperience in response to determining that the projected target locationcorresponds to a target location.
 18. The system of claim 17, whereinthe machine-readable instructions are configured to cause the one ormore processors to receive the initial location of the reference elementof the handheld object as projected on the two-dimensional plane. 19.The system of claim 18, wherein the machine-readable instructions areconfigured to cause the one or more processors to: determine a secondoffset along a second axis of the two-dimensional plane that compensatesfor distortion along the second axis associated with the arc in thethree-dimensional space by dividing the arc into a plurality ofadditional reference element zones, and wherein each additionalreference element zone of the plurality of additional reference elementzones is associated with an additional respective target zone of aplurality of additional target zones on the two-dimensional plane; anddetermine the projected target location of the handheld object based onthe initial location of the reference element of the handheld object onthe two-dimensional plane, the first offset, and the second offset. 20.One or more tangible, non-transitory, computer-readable media,comprising instructions that, when executed by at least one processor,cause the at least one processor to: receive an initial location of areference element of a handheld object as projected on a two-dimensionalplane; determine a first offset along a first axis of thetwo-dimensional plane that compensates for distortion along the firstaxis associated with an arc of movement of the reference element of thehandheld object in a three-dimensional space by dividing the arc into aplurality of reference element zones, wherein each reference elementzone of the plurality of reference element zones is associated with arespective target zone of a plurality of target zones on thetwo-dimensional plane; determine a second offset along a second axis ofthe two-dimensional plane that compensates for distortion along thesecond axis associated with the arc in the three-dimensional space bydividing the arc into a plurality of additional reference element zones,and wherein each additional reference element zone of the plurality ofadditional reference element zones is associated with an additionalrespective target zone of a plurality of additional target zones on thetwo-dimensional plane; determine a projected target location of thehandheld object based on the initial location of the reference elementof the handheld object on the two-dimensional plane, the first offset,and the second offset; and output a user interactive experience inresponse to determining that the projected target location correspondsto a target location.